Ghost penalties in nonconvex constrained optimization: Diminishing stepsizes and iteration complexity

We consider, for the first time, general diminishing stepsize methods for nonconvex, constrained optimization problems. We show that by using directions obtained in an SQP-like fashion convergence to generalized stationary points can be proved. In order to do so, we make use of classical penalty functions in an unconventional way. In particular, penalty functions only enter in the theoretical analysis of convergence while the algorithm itself is penalty-free. We then consider the iteration complexity of this method and some variants where the stepsize is either kept constant or decreased according to very simple rules. We establish convergence to δ−approximate stationary points in at most O(δ), O(δ), or O(δ) iterations according to the assumptions made on the problem. These complexity results complement nicely the very few existing results in the field.